The present invention relates to a moving image encoding device and, more particularly, to an improvement in code amount control in a moving image encoding device for encoding a plurality of objects.
A conventional moving image encoding device of this type is used to perform control to encode a moving image constituted by a plurality of objects within a predetermined code amount, as disclosed in, for example, Japanese Patent Laid-Open No. 11-225329 (reference 1).
FIG. 11 shows a conventional moving image encoding device for encoding a moving image constituted by a plurality of objects.
The first object data is input to an encoding section 1000. The first output of the encoding section 1000 outputs the first object code sequence, and the second output (first object generated code amount) is connected to a subtracting section 1003. The second object data and an output from the subtracting section 1003 are input to an encoding section 1001. The first output of the encoding section 1001 outputs the second object code sequence, and the second output (second object generated code amount) and third output (second object encoding information) are connected to a total VOP (Video Object Plane) target code amount determining section 1002. A unit time allowable code amount and the second and third outputs from the encoding section 1001 are input to the total VOP target code amount determining section 1002. The output (total VOP target code amount) of the total VOP target code amount determining section 1002 is connected to the subtracting section 1003. The output from the total VOP target code amount determining section 1002 and the second output from the encoding section 1000 are input to the subtracting section 1003. The output (second object VOP target code amount) of the subtracting section 1003 is connected to the encoding section 1001.
The operation of the encoding device in FIG. 11 will be described next.
The first object data is encoded by the encoding section 1000, and encoding is performed for each VOP. The first object code sequence as the encoding result is output. The code amount of each VOP generated upon this encoding is output as the first object generated code amount.
The total VOP target code amount determining section 1002 calculates a total VOP target code amount which is the target code amount required to encode all the VOPs on the basis of the encoding information and generated code amount which were obtained by the encoding section 1001 upon encoding of VOPs in the past and an externally supplied allowable code amount. The allowable code amount is a code amount for defining the upper limit of the total code amount of all the objects, e.g., a band or the capacity of data that can be recorded on a recording medium. Encoding information may be the average value of quantization steps.
A total VOP target code amount is a target value for the sum total of code amounts generated upon encoding of the VOPs of each object. As a method of determining a total VOP target code amount, for example, TM-5 (Test Model 5) of MPEG-2 (Moving Picture Experts Group 2) may be used. In this case, after an index representing the degree of complexity with respect to each of picture types, i.e., I, P, and B, from a past generated code amount and the average value of quantization steps, code amounts that can be assigned to GOVs (Groups of VOPs) are distributed in accordance with the obtained degrees of complexity, thereby calculating each VOP target code amount.
The total VOP target code amount output from the total VOP target code amount determining section 1002 is input to the subtracting section 1003. The subtracting section 1003 calculates a second object VOP target code amount as a target code amount required to encode the VOPs of the second object by subtracting the first object generated code amount which is a code amount spent to encode VOPs corresponding to the first object from the code amount output from the total VOP target code amount determining section 1002.
The second object VOP target code amount is input to the encoding section 1001. The encoding section 1001 encodes each VOP of the second object so as to match the second object VOP target code amount output from the subtracting section 1003, thereby creating the second object code sequence. In addition, the encoding section 1001 outputs the second object generated code amount representing the amount of codes actually generated upon encoding and encoding information to the total VOP target code amount determining section 1002.
Reference 1 discloses a device in which background image update information is treated as the first object, and an encoding section performs still image encoding such as JPEG. According to the device disclosed in reference 1, first of all, a background image is encoded as a still image and transmitted or stored, and then, a foreground object and the background update information are encoded and transmitted or stored. That is, the background image update information is encoded into a still image by the encoding section 1000, and the foreground object is encoded with the value obtained by subtracting the code amount generated upon this encoding from the total VOP target code amount. In this case, it is assumed that the code amount generated upon encoding of the background update information is sufficiently smaller than the total VOP target code amount.
However, the following problems arise in the above conventional moving image encoding device.
First, a target value for encoding of each VOP of the second object is determined by setting the VOP target code amount obtained by encoding control on the second object as a target value of a code amount assigned to all VOPs and subtracting the code amount actually generated upon encoding of the first object from the target value.
As the code amount required for encoding of the first object increases, the code amount assigned to the second object decreases, resulting in a considerable deterioration in decoded image quality. If the first object generated code amount is larger than the total VOP target code amount calculated by the total VOP target code amount determining section 1002, in particular, the second object VOP target code amount becomes a negative value. As a consequence, the VOPs of the second object cannot be encoded.
Second, the relationship between the first and second objects cannot be reversed.
Even if the roles of the first and second objects are preferably reversed during a moving image sequence, the conventional encoding section cannot reverse the relationship. The above encoding device encodes the VOPs of the first object first, and then encodes the second object with the remaining code amount. That is, the first and second objects can be regarded as master and slave objects, respectively. This relationship is fixed. For this reason, even if the master/slave relationship between background data and foreground data in encoding is preferably reversed as in a case wherein the size of a foreground object abruptly decreases, this relationship cannot be dynamically changed.
It is an object of the present invention to provide a moving image encoding device which can keep the image quality of an overall decoded image high, even if the generated code amount of a VOP of a main object temporarily increases, by reducing the influence of this increase on encoding of the remaining objects.
It is another object of the present invention to provide a moving image encoding device which can dynamically change the master/slave relationship between objects in encoding.
In order to achieve the above objects, according to the present invention, there is provided a moving image encoding device for encoding a plurality of objects, comprising first object encoding means for encoding input first object data within a unit time allowable code amount representing an allowable code amount in a unit time to create and output a first object code sequence, and outputting a first object generated code amount, code amount log storage means for storing the first object generated code amount, remaining object allowable code amount calculating means for adding up the first object generated code amount output from the code amount log storage means within each time interval with the unit time length to obtain a sum value, and subtracting the sum value from the unit time allowable code amount, thereby calculating a remaining object unit time allowable code amount, second object encoding means for encoding each VOP (Video Object Plane) of second object data by performing encoding control in accordance with an input second object VOP target code amount to create and output a second object code sequence, and outputting second object encoding information including at least one of quantization information, motion information, activity information, encoding error information, and object size information, and outputting a second object generated code amount, and second object VOP target code amount determining means for distributing the remaining object unit time allowable code amount to each VOP whose generation time is included in a time interval with the unit time length on the basis of the second object encoding information and the second object generated code amount, and calculating the second object VOP target code amount that is a target value of a code amount in encoding each VOP of the second object.